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ABSTRACT 



The Laboratory staff reported the unit development 
and field test activities in their effort to collect, analyze, 
synthesize, and disseminate new curriculum developments in secondary 
level social studies to school curriculum decision makers. The 
audience consisted primarily of social studies teachers and 
department chairmen. To further refine and evaluate their model 
information processing system, the staff defined the following unit 
objectives using the Bloom and Krathwohl taxonomies: 1) Cognitive 
Performance Objectives: to develop knowledge of programs, 
comprehension of relationships and implications, application, and 
evaluation of program information; and, 2) Affective Performance 
Objectives: to promote use of, and involvement with the unit. 3) The 

specific unit requirements were: parallel treatment of programs, 

accuracy, consistency, objectivity, flexibility, timeliness, 
relevance, uniqueness, and cost effectiveness. After an initial 
survey of 100 new programs, nine research based American government 
programs were selected for review in the multi-level unit including: 
introductory materials, project comparisons, program selection 
device, and detailed program reports. Three major field tests were 
conducted between February 1969 and spring 1970. The first was 
largely non- quantitative; the main and operational field tests were 
empirical studies with the populations, methodologies, and results 
enumerated here. It was concluded that the objectives were met. (SBE) 
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FOREWORD 



This is a final report on an educational development project. 
The outcomeof the project Is a tangible product that informs school 
people about nine new developments in the field of secondary social 
studies. The product is a multi-level, self-contained unit of infor 
mation suitable for use in any secondary school. The form of this 
report follows the pattern of the Information Unit (the product) 
itself. The first page is devoted to a quick chart-like "fact 
sheet," and the second section is a summary. The body of this 
report is presented and organized in a way comparable to the reports 
contained in the Information Unit itself. In other words, we have 
tried to practice what we preach: people need varying levels of 
information about an educational development; not all people need 
all the same information. 

The American Government Information Unit is part of a larger 
development effort by the Communication Program of the Far West 
Laboratory for Educational Research and Development. Similar re- 
views of new developments in other subject areas are available, 
underway, or planned. A case study on an Elementary Science Informa 
tion Unit is now available, and reports on units in individualized 
instruction and early childhood education are be'ing developed. An 
additional report summarizing the total design of the information 
system, its technology, and the research behind it is also planned 
in 1970. 

We hope that this report and its companions will provide a 
stimulus and model for more comprehensive reporting on the research 
and development of major educational products now underway across 
the country. The educational community and taxpayers in general 
have a right to know how public funds are being expended and how 
successful the results of those expenditures are. 

This project could not have been completed without the coopera- 
tion of a large number of people, including almost two hundred 
social studies teachers, principals, and others who participated in 
the field test evaluation of this Information Unit. A particular 
debt of gratitude must be expressed to Kenneth Moore and the Joint 
County School System, Cedar Rapids, Iowa, who assisted the Far West 
Laboratory staff in field testing the Unit in twenty- three eastern 
Iowa schools. 



C. L. Hutchins 
Berkeley, California 
January 1971 
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FACT SHEET 



Name: The American Government Information Unit 

Name of the Project and Development Agency: The Communication Pro- 

gram of the Far West Laboratory for Educational Research and 
Development 

Project Director; C. L. Hutchins, Ph.D. 

Senior Editors: Nancy Adel son and Sandra G. Crosby 

Content Area: A review of nine new secondary programs in American 

government 

Purpose: The Information Unit enables school personnel to decide 

whether to adopt (or reject) ti ese new, relatively well- 
developed curricula 

Target Users: Secondary school curriculum decision-making groups. 

Secondary audiences: pre- and inservice groups interested in 
updating their knowledge of new developments in secondary 
social studies curricula 

Format: A multi-level printed product containing introductory 

materials on each of nine programs. The product contains a 
decision- oriented introduction and detailed reports on each 
program 

Length of Use: At least three hours 

Readability Level (of the Information Unit itself): College 

graduate 

Availability: February 1971 

Cost: Estimated at $7.95 

Distributor: Lockheed Educational System 
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SUMMARY 



FINAL REPORT ON THE INFORMATION UNIT ON 
SECONDARY SOCIAL STUDIES PROGRAMS 



The number of new processes and products available to schools 
is increasing at a rapid rate. Many of these developments are 
relatively complex and difficult to understand in contrast to 
traditional materials. On the surface they are no more appealing 
than the great quantity of resources already available. Do 
teachers and administrators have enough time to find, gather, and 
process information about the new developments? Do researchers 
and developers adequately disseminate information that is readable 
and understandable? 

The staff of the Far West Laboratory believe that the answer 
to these questions is "no." As a result, they have developed the 
American Government Information Unit, The unit describes nine new, 
relatively we 1 1 -developed and researched programs intended for use 
in secondary school government courses. Use of the Information 
Unit shortens the work load of the school personnel who are responsible 
for reviewing these developments for possible adoption. The infor- 
mation is presented at several levels of detail (multi-level) and 
in a decision-focused format. 

The unit contains the following features: 

1. Introductory materials, including a discussion of the 
new social studies; an introduction to the unit; sunmary 
descriptions of the nine programs; a chart comparing the 
programs; and an aid for deciding which programs are most 
appropriate. 

2. Detailed Program Reports (one report for each program). 

The sections of the reports are: 

1. Goals and Objectives 

2. Content and Materials 

3. Classroom Strategies 

4. Student and Teacher Prerequisites 

5. Implementation: Requirements and Costs 

6. Program Development and Evaluation 

7. Project History 

These materials are all packaged in a small, mailable box. 
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The programs reviewed are: 

1. High School Curriculum Center in Government Project 

2. Justice in Urban America Series, Law in American Society 
Foundation 

3. Selected Episodes, Sociological Resources for the Social 
Studies 

4. Secondary Social Studies Program, Lincoln Filene Center 

5. Utah State University Social Studies Project (A Curricu- 
lum Focused on Thinking Reflectively About Public Issues) 

6. AEP Public Issues Series (Harvard Social Studies Project) 

7. Holt Social Studies Curriculum 

8. The Amherst Project 

9. Educational Research Council of America Social Studies 
Program 

The results of the evaluation of the Information Unit showed 

that: 

T. Over 70 % of the subjects could achieve scores of 50% or 

higher on tests of knowledge and comprehension of programs. 

2. Over 50% of the subjects could make decisions regarding 
at least six of the nine programs. 

3. Subjects applied the information to their own needs. 

4. Subjects experienced a statistically significant increase 
in their own estimation of their knowledge about the 
programs. 

5. Average ratings by subjects placed the Information Unit 
at 5.0 or higher (toward the positive end) of seven-point 
scales of "useful" and "easy to use." 

6. Given a list of possible resources of curriculum infor- 
mation, subjects indicated that they preferred the Infor- 
mation Unit to all other secondary sources of comparable 
information (hiring consultants, professional meetings or 
conventions, and journals) and almost equivalent to all 
primary sources (workshops using the new curriculum 
materials, site visits to innovative projects, and con- 
versations with professionals whose judgments they 
valued). 

The unit was developed by the staff of the Communication Pro- 
gram of the Far West Laboratory for Educational Research and Develop- 
ment, Dr. Paul D. Hood, Director. Dr. C. L. Hutchins was the project 
di rector. 
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1.0 GOALS AND OBJECTIVES 



1.1 Long-range goals . 

The Amen'.accn Government Information Unit describes nine new 
secondary social studies programs. These programs are research- 
based; they represent the best-developed instructional programs 
available today. The Information Unit is intended to aid in 
bridging the gap between these new developments and the nation's 
schools. The goal is to provide information that will lead those 
people using the unit to decide to adopt, adapt, or reject the 
programs. The goal is not to induce adoption where the use of 
these programs would conflict with existing school requirements 
or resources. 

1 .2 Program objectives . 

The first objective of the Information Unit is to produce 
knowledge and understanding about the new programs— especial ly facts 
about each program, the relationship of these facts to general 
trends, and lists of criteria that school personnel have indicated 
are required to judge the new programs. Second, and most important, 
the Information Unit leads users to evaluate the new programs; they 
should apply information to their own situation and decide whether 
to adopt, adapt, or reject each program. The Information Unit does 
not contain evaluations of the programs; instead it enables the 
users to evaluate the programs using their own criteria. 

The Information Unit also has affective objectives: users will 
receive it, become involved with it, and value it. 

Other objectives of the Information Unit include the requirements 
of parallel treatment of programs, accuracy of facts, consistency 
among components, objective treatment of facts, timeliness, flexi- 
bility, relevance, uniqueness., and cost-effectiveness. 

Unstated, but implied, is another objective: to lead users' to 
value the products of research and development and the technology 
behind them. 

1 . 3 Detailed objectives . 

Following the B1 oom-Krathwohl taxonomies, and making a distinc- 
tion between performance (terminal) objectives and enabling objec- 
tives, the Laboratory staff has defined the following objectives: 
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Performance Objectives (cognitive), (a) Knowledge: After using 
the Information Unit (IU), \.he user will know major character- 
istics which differentiate the programs; (b) Comprehension: 

After using the IU, the user should identify relationships 
among programs which enable him to derive useful implications 
about programs; (c) Application: After using the IU, the user 
will apply information about programs to his own situation; 
and (d) Evaluation: After using the IU, the user will judge 
whether he would adopt, adapt, or reject programs in the IU. 

The Laboratory staff did riot define analysis or synthesis 
objectives iri the cognitive domain. They assumed that it would be 
difficult if not improper to provide an explicit system for evalua- 
tion; each user's needs and resources were judged to be so unique 
that no common set of procedures for evaluating the programs seemed 
appropriate. Instead, the Information Unit sought to provide those 
facts, etc., necessary for the user's own analysis/synthesis 
scheme — either explicit or implicit. 

Performance Objectives (affective), (a) Attention: If the IU 
is available to the user, he will attend to it; (b) Response: 

The user will respond to (become involved with) the IU; (c) 

Value: After using the IU, the user will value it above other 
secondary sources of information about programs and about as 
highly as he values most primary sources. 

Enabling Objectives, (a) Parallel Treatment of Programs:- It 
is almost axiomatic that programs be treated in a parallel 
manner; (b) Accuracy of Information: The information provided 
will be accurate; (c) Consistency of Information: All infor- 
mation about each program will lead the users to congruent 
conclusions about the program; (d) Objective Treatment: All 
information in the IU must be supported by the ability of a 
number of judges to agree that the information is true; (e) 
Timeliness of Information: The time span between development 
of the unit and use of the unit will not significantly diminish 
the value of the information; (f) Flexibility: The unit should 
be usable by various types of individuals in a number of dif- 
ferent kinds of settings; (g) Relevance: Those areas of highest 
priority or relevance should be treated first; (h) Uniqueness: 

The unit will not represent a duplication of effort; (i) Cost- 
effectiveness: The user should perceive that the reward for 
using the unit justifies the cost and time involved. 

1.4 How are the detailed objectives derived from the goals? 

Each of the cognitive and affective performance objectives, and 
the objective of cost-effectiveness of the development, were translated 
into behavioral objectives; conditions were specified and standards 
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set. All the enabling objectives, with the exception of cost- 
effectiveness, passed or failed on the basis of expert judgment, 
the opinions of program developers, or logical analysis. An 
abridged statement of the behavioral objectives follows: 

1. COST-EFFECTIVENESS: After using the IU, subjects will 

rate their use of the IU on three scales of reward 
(personal benefit, helpfulness, learning) and three 
scales of cost (time, money, effort). Subjects will 
score the IU 1.0 or greater, based on a reward/cost 
rati o J 

2. COGNITIVE OBJECTIVES: 

'a. (Knowledge). After using the IU, subjects will, 

without guessing, correctly identify as true or false 
significantly more than 50% of a set of statements 
describing major characteristics of programs. 

b. (Comprehension). After using the IU, subjects will, 
without guessing, score significantly higher than 50% 
correct when indicating agreement or disagreement with 
statements about programs which represent implications 
based on facts about programs they reviewed; 

c. (Application and Evaluation). After using the IU, 
subjects will identify those programs which are ac- 
ceptable in light of their five most important 
needs, resources, or constraints. Subjects will 
make positive decisions (pilot test, adopt, etc.) 
for programs which meet more than half of their five 
most important needs and negative decisions (reject) 
for programs which do not. 

d. (Evaluation). After using the IU, subjects will indi- 
cate what course of action they would taki regarding 
the IU programs. Fifty percent of the subjects will 
decide to "send for sample materials," "pilot test," 
"adopt," or reject at least six of the nine programs. 



Ifhe standards specified are minimal rather than optimal standards. 



3. AFFECTIVE OBJECTIVES (Attention, Response and Value): 2 

a. After using the IU, subjects will indicate their 
agreement with statements regarding the different 
pieces of the IU. Their ability to respond will be 
interpreted as ability to differentiate the pieces. 
Eighty percent or more subjects will indicate agreement 
or disagreement with each statement. 

b. After using the IU, subjects will indicate whether 
they would be willing to recommend to their schools 
the use of IU 1 s in other secondary areas. Eighty 
percent of subjects will indicate that they would 

be willing to recommend IU's in other secondary areas. 

c. After using the IU, and given a list of possible 
primary and secondary sources of information about 
programs, subjects will indicate on scales of "use- 
ful" and "easy to use" that the Information Unit 

is superior to all secondary sources. 3 



2 The basic measures of attention and response are supplemented by a 
number of "unobtrusive" measures. No one measure seems adequate to 
observe attention or response; thus, the staff has looked for 
multiple evidences to determine whether subjects were attending and 
responding to the IU. If the existence of attention and response 
can be confirmed by a number of measures, we increase our certainty 
that they have, in fact, occurred. 

^A complete description of these objectives along with sample items 
can be obtained from the field test reports available at the Far 
West Laboratory for Educational Research and Development. 



2.0 CONTENT AND MATERIALS 



2 . 1 What programs were selected for review and inclusion in this 
Information Unit? What general subject or discipline do the 
selected "programs cover? 

Nine secondary social studies programs in the area of American 
government were included for review: 

The Amherst Project 

Project Director: Richard Brown, Committee on the Study 
of History, Hampshire College 
Publisher: Adaison-Wesley Publishing Company 

Educational Research Council Social Science Program (ERCSSP) 
Project Director: Raymond English, Educational Research 
Council of America 
Publisher: Allyn and Bacon, Inc. 

Harvard Social Studies Project/AEP Public Issues Series 
Project Director: Donald Oliver, Harvard University 
Publisher: American Education Publications 

High School Curriculum Center in Government Project (HSCCGP) 
Project Director: Howard Mehlinger, Indiana University 
Publisher: Ginn and Company 

Holt Social Studies Curriculum (HSSC) 

Project Director: Edwin Fenton, Carnegie-Mel Ion University 
Publisher: Holt, Rinehart and Winston, Inc. 

Justice in Urban America Series 

Project Director: Robert Ratcliffe, Law in American 
Society Foundation 
Publisher: Houghton Mifflin Co. 

Lincoln Filene Center (LFC) Secondary Social Studies Program 
Project Director: John Gibson, Lincoln Filene Center for 
Citizenship and Public Affairs 
Publisher: Lincoln Filene Center 

Sociological Resources for the Social Studies (SRSS) (Selected 
Episodes) 

Project Director: Robert Angel 1, SRSS 
Publisher: Allyn and Bacon, Inc. 
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Utah State University (USU) Social Studies Project (A Curricu- 
lum Focused on Thinking Reflectively About Public Issues) 
Project Director: James Shaver, Utah State University 
Publisher: Houghton Mifflin Co. 

These programs were sel 'd because their subject areasi are 
particularly relevant to the . ..idy of American government. Their 
subject coverage ranges from law, legal issues, American political 
behavior, political systems, government institutions, and public 
issues to American history and sociological problems and issues. 

2.2 What grade levels and difficulty levels are covered? 

All of the programs are designed for use at the secondary 
school level (grades 9-12). Each developer recommends the grade(s) 
in which use of the program seems appropriate. In addition, the 
Far West Laboratory (FWL) has computed a readability grade level 
for each program, based on analysis of sample passages from student 
materials. 4 These grades and readability levels are as follows: 





TABLE 1 






Program 


Grade Level 

(As recommended by the 
developer) 


Readability Level 5 
(As computed by the Laboratory 
staff) 


Amherst 


Any secondary grade 


12th grade 




ERCSSP 


Grade 9 


10th grade 




Harvard 


Any secondary grade 


8th grade 




HSCCGP 


Any secondary grade 


11th grade 




HSSC 


Grade 9 


11th grade 




Justi ce Series 


Grade 9 


9th grade 




^The method for 


determining readability 


involved use of the 


Dale- 



Chall formula. This formula takes into account average sentence 
length and percentage of unfamiliar words (i.e., words not appear- 
ing on the Dale list of 3,000 easy words) in the sample passages. 
The formula produces a raw score which is then corrected to the 
corresponding grade level. 

5 A11ow for a standard deviation of one year from the grade level 
specified. 
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Program 



Grade Level 



Readability Level 



LFC 


Any secondary grade 


8th grade 


SRSS 


Any secondary grade 


9th grade 


USU 


Grades 11, 12 


10th grade 



All of the programs have also been classified by FWL as appro- 
priate for certain student ability levels. The classification of 
each program is based on (a) the range in readability levels of 
the student materials*, (b) curriculum expert and/or staff assess- 
ment of 'the program's grade and student ability levels; and (c) 
the program developer's recommendation of grade and ability levels. 
Two programs, the Justice in Urban America Series and the LFC 
Secondary Social Studies Program, are classified as appropriate 
for "below average to average" ability students. Programs appro- 
priate for students of "average to above average" ability include 
SRSS, HSCCGP, ERCSSP, Harvard, and the USU Project. Amherst and 
HSSC are classified as suitable for "above average" ability students. 

The target audience of the Information Unit is personnel 
responsible for selecting and implementing social studies programs 
in secondary schools. Although curriculum coordinators and princi- 
pals are included in this group, the audience consists primarily 
of social studies teachers and department chairmen. Research con- 
ducted as part of the development of the Information Unit produced 
no evidence that this audience could be further subdivided or 
differentiated. There is no basis in the data to confirm that any 
single group of district- or school-level personnel could be iden- 
tified as the "decision-makers." FWL did learn from field testing 
and consultation with school personnel that at the secondary 
school level teachers and/or department chairman frequently have 
the authority to select some materials and make some curriculum 
decisions without seeking the approval of higher-level school or 
district personnel. 

2.3 How complete is the subject coverage of the Information Unit 
and the selected programs? 

The Information Unit was not intended to cover all social 
studies programs available. An initial survey of the latest devel- 
opments in the field revealed nearly 100 new programs, which 
focused on a variety of subjects, including' history , economics, 
geography, government, anthropology, and interdisciplinary studies, 
among others. Staff size, financial resources, and other constraints 
required that the review be limited initially to one area. A 
logical starting place seemed to be American government, which is 
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an area covered in every high school curriculum. The nine programs 
selected as relevant to the study of American government were judged 
to be the best examples of research and development in that area. 
Standard textbook programs, audiovisual and other resource materi- 
als, and supplementary simulation materials were excluded during 
the review. (See 2.6 for the procedures used to select the 
programs. ) 

The subject area of each program in the Information Unit is 
relevant to the study of American government. The programs' 
coverage of the area varies, however. Some of the programs are 
designed as complete, sequential, secondary curricula, with indi- 
vidual units or courses focusing on government, political science, 
or related areas (HSSC, ERCSSP). Units in the LFC instructional 
programs can be used either as a sequential curriculum or as sup- 
plements to an existing curriculum. Three of the programs are 
self-contained courses: two (Justice Series and HSCCGP) are a full 
year in length and designed primarily for grade 9; one (USU) is 
a semester in length and is designed primarily for grades 11 and 
12. Units from the Justice Series can also be used as supplemen- 
tary material. The three remaining programs (SRSS, Amherst, and 
Harvard) provide supplementary units which focus on particular 
topics, problems, social phenomena, or issues. They are designed 
to replace or expand existing curricula, courses, or textbooks in 
the 9-12 grade range. 

2.4 What concepts and processes are to be learned in the programs 
reviewed ? 

The new social studies programs emphasize learning about issues 
and ideas and how to deal with them, rather than acquiring factual 
information. The developers of these programs contend that, tradi- 
tionally, students have been asked to memorize names, facts, and 
dates and to study the conclusions of social science investigations. 
Their programs, on the other hand, encourage students to study and 
use the processes of social science investigation in order to gain 
understanding of the nature of the social sciences. Students of 
the new social studies, therefore, are generally involved in 
inquiry— they are encouraged to ask questions, discover for them- 
selves, and apply skills and processes used by social scientists. 
Major processes stressed in the programs include questioning, 
analyzing, formulating hypotheses, drawing conclusions, and valuing. 

The new programs also provide students with a framework of 
concepts, theories, and generalizations which give relevance to and 
help to organize the factual information they learn. 



Content in the new programs may be organized around concepts, 
such as power, leadership, or liberty; topics or themes, such as 
the presidential election of 1964 or the voter in American society; 
or problems -and issues, such as the right to counsel or freedom of 
assembly. 

2 . 5 What concepts and processes are to be learned from the Informa - 
tion Unit itself ? 

Learning of specific concepts and processes was not a terminal 
objective of the Information Unit. Although field test participants 
were expected to know and comprehend certain major characteristics 
of the programs they reviewed (such as content perspective, grade 
level, teaching strategy, evaluation and training requirements, e»*d 
program history), their ability to do so was a measure of their 
achievement of the intermediate, rather than terminal, objectives 
of the Information Unit. 

2 . 6 What rationale and procedures were used to select or identify 
the programs included in the Information Unit ? 

To be included in the Information Unit, a project had to meet 
all of the seven selection criteria described below. First, the 
purpose C'f the project had to be to produce curriculum materials 
for students. Such materials could include sets of readings, 
study units, texts, and/or casebooks which were organized as sup- 
plementary units or complete curricula or courses. Second, the 
materials had to focus on subject areas which could be used di- 
rectly in American government, problems of democracy, civics, po- 
litical science, politics, or public issues courses. The subject 
area could include combinations of topics, such as national, state, 
and/or local government; public opinion, pressure groups, and 
politics; citizenship and political action; historical background 
and basic concepts; foreign policy; economic problems; and/or 
social problems. Third, the materials had to be recommended for 
use at the secondary school level in grades 9, 10, 11, and/or 12. 
Fourth, the material had to have undergone an adequate formal 
development and testing procedure, comparable to the cycle followed 
by FWL, and also produced field test evidence that the materials 
could be successfully employed by teachers and students in actual 
classroom situations. This procedure had to include classroom 
trial and then revision of materials, based on data gathered from 
the trial, before the materials were released for general use. 

Fifth, the materials had to be ready for general distribution by 
the beginning of the 1970-71 school year. Sixth, the minimum time 
required to use the materials had to be approximately two school 
weeks or 10 to 20 classroom hours. Seventh, the materials had to 
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be accompanied by instructions about the teaching strategy implicit 
in or appropriate to the materials. 

Staff review of professional literature, reports, and journals, 
staff interviews with San Francisco Bay Area social studies educators, 
and consultation with nationally recognized experts in the field 
identified initially about 35 social studies projects as potentially 
appropriate for inclusion in the Information Unit. After applica- 
tion of the criteria described above, nine projects from that list 
subsequently qualified for inclusion. Bach of them met all of the 
seven selection criteria. 

Although other programs might be considered appropriate for 
inclusion in the published edition of the Information Unit, the 
number will be restricted to those presently included. This re- 
striction seemed justifiable for two reasons: (1) addition of 
more programs and, consequently, preparation of descriptive materi- 
als on them, would delay the release of the Information Unit beyond 
the 1970-71 school year.- and (2) the second-generation information 
system now under development is designed to incorporate such 
programs--it has a built-in capability to add new programs as they 
become available and will, eventually, subsume the present Infor- 
mation Unit. 

2.7 What is the general form and organization of the Information 
Unit? 



The American Government Information Unit follows a model 
initially conceptualized for the first information system developed 
by the Far West Laboratory. Briefly, that format includes three 
levels of information: (a) the "Survey Level," a broad survey of 
trends in secondary social studies; (b) the "Intermediate Level," 
charts and summaries that describe the most noteworthy features of 
each program and its classroom activities; (c) the "Detailed Level," 
reports on each program organized under the headings Goals and 
Objectives, Content and Materials, Classroom Strategy, Student and 
Teacher Prerequisites, Implementation Requirements and Costs, and 
Project History. The organization and general content format of 
the "Detailed Level" closely follows that of this report. The 
model specifies that the user should be able to narrow successively 
his choice of programs as he completes each level (in descending 
order). 

The organization of content within each level is topical. A 
chronological order (following the steps taken by the developers) 
was tried and rejected early in the trials of the Unit. The topical 
order selected was based on as much empirical evidence as could be 
found. Where empirical justification could not be found to support 
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a particular ordering of topics, the judgment of the development 
staff was used. 

2.7.1 What is the sequencing of the subdivisions ? 

As indicated, the units are sequenced so that each successive 
level of information is more inclusive and comprehensive than the 
last. Some repetition of information at each level is found in 
the next highest level. 

2 . 8 How independent are the various elements or subunits ? 

Because of the repetition of information at each level in the 
next most detailed level, any level may be read first; it is not 
essential that a specific pattern be followed. 

However, in order to save time, it is recommended that the 
user start with the first level and, on the basis of the informa- 
tion in that level, attempt to omit programs that do not meet his 
requirements. The assumption is that he will prefer not to read 
all of the material at the "Detailed Level" if he can omit pro- 
grams unsuitable to his needs on the basis of briefer information 
at preceding levels. 

2 . 9 What materials are provided at each level? In what quantity 
are they provided ? 

All materials in the Information Unit are in print form. 
Because classroom activities and materials in the social studies 
programs are not as highly visual as, by contrast, are those in 
science laboratory programs, the staff judged that an audiovisual 
presentation on each program was not essential for making decisions 
among the nine programs. This judgment was confirmed by field 
users during field testing. In addition, because it was to be 
used at the secondary school level, it was assumed that the unit 
would be used primarily by a small number of individuals rather 
than groups, so that printed material would be more convenient to 
use. This assumption was confirmed early in the field testing of 
the unit. 

The levels of information in the unit include the following 
materials : 

Summary and Intermediate Levels . Information presented at 
this level was designed: (a) to introduce the user to and inform 
him about the purpose and content of the Information Unit; (b) to 
suggest ways to review information in the unit; (c) to provide a 
general preview of the nine programs; and (d) to enable the user 
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to differentiate among the programs enough to select a few to 
review in detail. 

A cartoon-and-print booklet, called the Preface, was developed 
to serve these purposes. This booklet contains a series of cartoons 
which briefly introduce the unit, describe the types of information 
provided, and explain how to use it; a Screening Aid, an optional 
part of the Preface, which enables the user to identify programs 
having his set of preferred characteristics; a Preview Chart, 
listing major characteristics of each program, such as content 
perspective, suggested curriculum use, grade level, readability 
level, etc., which can be used for initial review and comparison 
of the programs; nine Previews, each a brief narrative description 
revealing major content areas, specific topics and titles of 
materials, teaching strategy, and typical classroom activities of 
one program; and a Glossary, which defines some of the terms and 
characteristics used in the Preface and throughout the unit. 

Detailed Level . This level provides comprehensive information 
about each program in the unit. This level is intended to provide 
the user with the detailed information he needs to be able to make 
a decision or recommendation regarding the appropriateness of each 
program for field testing, adaptation, adoption, or rejection in 
his school. 

There is only one form of information at this level--the 
printed Report. Nine Reports, one on each program, are included in 
the unit. Each one is divided into seven sections of detailed 
information focusing on: (1) Goals and Objectives; (2) Content and 
Materials; (3) Classroom Strategy; (4) Student and Teacher Prerequi- 
sites; (5) Implementation: Requirements and Costs; (6) Project 
Development; and (7) Project History. In addition, there is a 
page of Basic Facts about the program at the front of each Report. 
This page repeats the summary information covered in various parts 
of the Preface. 

The Operational Field Test Form of the Summary and Detailed 
Level were packaged in a 9" x 11" x 2" box, with the top side open 
for easy removal of booklets. The box could be stored on regular 
size library or bookcase shelves. Because of difficulties in find- 
ing a publisher who would comply with this format, however, the 
final release form will constitute a single, bound volume. 



3.0 INSTRUCTIONAL STRATEGY 



3.1 What is the instructional strategy of the Unit? 

The instructional strategy underlying the Information Unit can 
be stated as a series of assumptions: 

1. The curriculum-adoption,. decision-making practices of 
schools are extraordinarily varied; no single model of how needs 
are assessed, goals and objectives identified, information about 
alternatives gathered and processed, and decisions made can be 
established among school organizations. It is assumed that the 
processes in most cases involve multiple, reoccurring encounters 
with needs, goals, and information--rather than a uni dimensional 
model of the processes. 

2. It is assumed that a great variety of personnel are 
involved— ranging from parents and students to teachers and 
administrators. It is also believed that the variation in infor- 
mation-processing behaviors is as broad within groups as it is 
among groups. In general, the process is assumed to involve both 
group and individual acts ranging over a period of time. 

3. It is assumed that the types of information needed can 
be identified in scope and character but that the general orcler 
and form needed will vary greatly from one individual to another 
and from one group to another. 

4. It is assumed that motivation to want and need the type 
of information provided by the unit must exist before coming to 
the unit; the unit cannot induce review and rational consideration 
of new alternatives leading to adoption unless the user is inclined 
in this direction. 

5. It is also assumed that a mailable package of information 
about new curriculum projects is a necessary but not sufficient 
stimulus for rational decision-making; other types of materials and 
contacts must also be present to lead to the terminal behaviors of 
adoption, adaptation or rejection. 

Most of these assumptions are untested and research about them 
is lacking. As a result, the choice of strategies was either to 
make a considerable initial effort to find evidence supporting or 
rejecting these assumptions or to begin development and attempt 
to establish supporting information as we went along. The latter 
course of action was the one adopted. In this specific case, the 
resulting design included these instructional characteristics: 
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1. Except for the initial instructions , the sequence for 
using the unit was not linear or discursive; entry to the body of 
the content was by multiple paths suiting the variety of styles 
used by individuals and organizations. Hence, the need for each 
major part to be relatively independent of the others. Redundancy 
exists where the prerequisite information is necessary to under- 
stand particular cognitive elements. 

2. The information must cover a range wide enough to fit 
multiple needs; if an error is committed it should be on the side 
of comprehensiveness rather than exclusiveness. 

3. Though motivation is presumed, the unit should provide 
sufficient "persuasion" that the user already inclined toward the 
use of such an approach will be induced to believe that the 
product is superior for his needs. 

4. The unit must be flexible enough to be used as an auxiliary 
to other materials and in a variety of situations involving its use 
by "linking agents," peers, and such formal channels as pre-service 
education. 

3.2 What various patterns of use are suggested? 

The Operational Field Test version of the Preface suggests 
several alternative plans for individuals reviewing information 
provided in the unit; however, it recommends that groups using 
the unit devise a review plan unique to their particular situation 
and needs. 

3.3 What preparation is required to use the Information Unit? 

In the Preface to the Information Unit, suggestions are made 
as to how additional information and activities might be gathered 
to enhance the use of the unit. In general, however, the use of 
the unit requires nothing beyond the information provided. 
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4.0 USER PREREQUISITES 



4.1 What skills, background or training are needed to use the 
Information Unit ? 

The Information Unit was designed so that users would need 
no special background or skills in order to use it successfully. 
During the course of field study, little or no relationship could 
be found between use of the unit and the following variables. 

1. Urban/Rural Setting 

2. School Size--Average Daily Attendance (range: less than 
200, between 1,000 and 4,999) 

3. Grades in School--All secondary grades 9-12 vs. senior 
grades 11-12 only incorporated in school 

4. Purpose in Using the Unit (range: information resource, 
decision-making, preservice and inservice training) 

5. Education Level (range: Student to Ph.D.) 

4.2 What is the reading level of the Unit? 

Laboratory staff used the Dale-Chall formula to determine the 
readability level of the Preface and the nine Reports in the Infor- 
mation Unit. The readability level ranged from 13th grade to 
college graduate. The average readability level for all 10 book- 
lets was college graduate level. 
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5.0 IMPLEMENTATION: REQUIREMENTS AND COSTS 



5* 1 What type of physical setting is needed to use the unit ? 

It can be used under a variety of conditions: by one person 
at a desk or by small groups in an office or classroom. 

5.2 What audiovisual equipment must be provided? 



None. 

5.2 What material must be Purchased to replace consumable items ? 

Initially, the Prefacs was intended as a handout; three were 
packaged with the unit. However, in the final release form the 
unit will be a single item. 

5.3 What type of classroom organization is required? 

The question is not applicable. 

5.4 What kind of administrative supervision is needed or 

recommended ? ” 

It is strongly recommended that administrators be involved 
with teacher groups in using these materials. In some field test 
situations where administrators were not present, the participants 
felt very frustrated because they felt that if they had wanted to 
adopt one of the new programs, no positive action could be anti- 
cipated without administrative understanding, support, and con- 
currence. 

5.5 Is inservice training necessary to use the Information Unit ? 

No, the unit is self-contained, self-explanatory, and requires 
no training for use if directions are followed. (It should be 
noted that the unit can be used as part of an inservice training 
course for secondary teachers.) 

5.5 How critical is a public relations or information effort 
prior to or during use of the unit? ' 

None is required. It should be noted that the Information 
Unit can be used as part of a community or school information 
effort if one of the programs described is to be adopted. 



5.7 What is the availability of the unit? 



The unit will be available in February 1971. 

5.8 Does the unit require materials; developed by others ? 

No, but it should be noted that the programs described by 
the Information Unit are all available and actual teaching materials 
can be obtained directly from the publishers of these programs to 
supplement the use of the Information Unit in the decision-making 
process. 

5.9 What does the Information Unit cost? 



The cost is estimated at $7.95. 

5.10 What descript ive information about the Information Unit is 
available ? — “ 

A handout can be obtained from the Far West Laboratory. 
Copies of this report are also available. 
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6.0 PROJECT DEVELOPMENT AND EVALUATION 



6. 1 Why was the project initiated ? 

The developers of the American Government Information Unit 
proposed to collect, analyze, synthesize, and disseminate informa- 
tion about new curriculum developments in secondary level social 
studies to school personnel responsible for curriculum decision- 
making. The need for this type of processed information seemed 
clear. Numerous new products have become available within the 
past decade; yet school personnel often do not have the time to 
identify them and then gather and process information to ascertain 
which are suitable to their needs. Furthermore, developers, as 
well as other agencies responsible for research and development in 
education, have not, in the past, planned adequately for dissemina- 
tion of information in a format and style acceptable to and 
feasible for school personnel. 

The field of social studies was selected for this project 
because the need for information about current developments in the 
field was particularly acute. This decision was supported by the 
results of two surveys conducted within the area served by the Far 
West Labr>v'A;ory. One study (conducted by the ESEA Title III Pace 
Center in Solano, Sonoma, Napa, and Marin counties in California) 
showed that social science tied for second among 20 curriculum 
fields which had the highest amount of divergence between what is 
and what should be. In the other Bay Area study, the Laboratory 
found that social science was third highest among ten fields of 
which teachers and principals expressed interest about new develop- 
ments. 

It should be noted also that development of the American Govern- 
ment Information Unit was initiated to further refine and evaluate 
the model of the information processing system under development 
by the Laboratory. In particular, the Unit was intended to deter- 
mine the applicability of the model for a secondary school level 
audience and to determine the feasibility of and receptivity to 
an all-print, rather than multi-media, information format. 

6 . 2 What is the research base of the development ? 

At the outset of the development of the. Information Unit, very 
little was known abouc what kind of information, arranged in what 
order and in what form, school personnel need to make what decision. 

A significant body of social science research and knowledge has 
been developed in the area of dissemination and adoption of develop- 
ments including those in education, but little is known about the 
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information required to activats such disssmi nation-adoption 
cycles. Probably the area with the most sophisticated base for 
such a development is advertising. Unfortunately, not only is 
much of the practice in advertising relatively intuitive and judg- 
mental, but what research has been done focuses on activating 
psychological principles and processes that are inappropriate to 
the rational decision-making process prescribed for the Information 
Unit. In short, no claim can be made that a research base existed 
for the development of the product. (Most of the untested assump- 
tions behind the unit are listed in Section 3.0 of this report.) 

The form of the unit was influenced somewhat by experience 
with an earlier Information Unit, the Elementary Science Information 
Unit. For example, experience with that unit indicated that users 
were less interested in some kinds of information (e.g. project 
history) than in others (e.g. teaching strategy). However, the 
difference between the two units and the proximity in time of their 
development did not allow too much impact upon one by the other. 

6.3 Through what stages has the development gone? What were the 
important conditions at each stage? 

The development of the American Government Information Unit 
followed the cycle specified for developing all major products of 
the Far West Laboratory. The major steps included: 

Initial Planning. Review of the literature, initial planning, 
consultation with a panel of experts in the field and site visits 
to projects were initiated in June of 1968. Conceptualization of 
the model and prototype form were completed in the Fall. An initial 
development plan, consultant records, description of proposed 
models, and other working papers document this early phase. 

Pre liminary Pnoduot Dev&lavTnent. Preparation of materials 
for preliminary field testing Degan in December 1968, and con- 
tinued through the end of January 1969. The preliminary form 
consisted of the following materials at each level (see Section 
2.0 for explanation of nature and purpose of each level). 

The summary level included: (1) an audio tape presentation, 
the "Overview," discussing background, trends, and materials rele- 
vant to social studies curriculum reform and development; (2) a 
printed "Program Characteristics Guide," listing alternative charac- 
teristics which could be used in selecting programs to review in 
detail; (3) a printed, hole-punch set of sheets, the "Program 
Selection Device," used to identify and select programs having 
specified characteristics; and (4) a printed booklet, "Program 
Summaries," containing one-page narrative descriptions of each 
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program with emphasis on significant features. 

The detailed level offered complete "Reports" on three pro- 
grams (SRSS, HSSC, and HSCCGP). Each report provided detailed 
information 'about one program in terms of 14 categories, which were: 
Goals and Objectives, Philosophy, Content, Organization of Content, 
Materials, Administrative Requirements, Evaluation, Organizational 
Background, Costs, Research and Development Procedures, Student 
Testing and Evaluation, Classroom Procedures, Teacher Prerequisites 
and Preparation, and Student Prerequisites. The 14 sections of 
information were organized differently for each report: in one, the 
sections were arranged according to a "teacher's model" for order 
importance of information needed; in the second, according to an 
"administrator's model;" and the third, according to a "chronological 
model." The various models were based upon hypothetical differences 
projected by the project staff. The purpose of presenting three 
different models or organizations of information was to examine 
user priorities regarding importance and order of information 
needed. 

Preliminary Field Test (PFT). The Preliminary Field Test 
took place in February 1969, with 19 field test subjects from the 
Bay Area. All subjects were active, "innovative" teachers and 
administrators with some post graduate education. The test was 
administered in the Laboratory. 

Subjects judged that the objectives of the unit were worth- 
while and that the approach being taken was likely to achieve the 
objectives. Subjects indicated that all parts of the unit, except 
the Overview tape, were necessary and interesting. The staff con- 
cluded that the unit had a strong potential for success and that 
the Overview should be discarded. 

As part of the PFT, an experiment was conducted to determine 
what order of information subjects preferred in the reports. The 
sequence indicated by subjects was used in reorganizing the reports. 

Plain Form Development. Preparation of the performance or 
"main" form of the Information Unit began in March, 1969, concurrent 
with the analysis of preliminary field test results. Development 
continued through April 1969, and resulted in the following 
elements: 

At the summary level there were: (1) two versions of a Preface, 
one in print and the other in cartoon-and-print. Each Preface 
provided an introduction to the unit, described its contents, and 
suggested alternative ways to use it (this replaced the PFT audio- 
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tape Overview); (2) a "Program Selection Guide," a booklet combining 
the former Program Characteristic. Guide and a revised, "carbonized" 
form of the PFT Program Selection Device; and (3) a Program Summaries 
booklet, containing the one-page narrative descriptions of each 
program and,- in addition, a chart summarizing major features of 
all programs. 

The detailed level again consisted of reports--on six of the 
programs. Based on PFT results, the number of sections of infor- 
mation in each Report was reduced to nine; these sections were 
arranged in the following order: (1) Goals and Objectives; (2) 

Content and Materials; (3) Classroom Procedures; (4) Origin and 
History; (5) Research and Development Procedures; (6) Student and 
Teacher Prerequisites; (7) Evaluation; (8) Administrative Require- 
ments; and (9) Costs. In addition, packets of original materials, 
including project proposals, evaluation reports, journal articles, 
and student and teacher manuals, were compiled on two additional 
programs to field test the user preference for organized descriptive 
booklets vs. unorganized, primary materials. 

Main Field Test (MFT) . The Main Field Test was conducted in 
the field. A purposive sample of 48 respondents from nine school 
sites was selected to approximate the following conditions; (a) 
secondary school personnel within a 250-mile radius of the Labora- 
tory; (b) positions as secondary school social studies teachers, 
principals, curriculum specialists and/or administrators ; (c) 
interest in innovation. Testing took place in the Summer of 1969. 

The IU was successful in achieving its objectives. Respondents 
were able to make acceptable decisions about most IU programs. In 
addition, the IU reduced the need for further information and led 
subjects to review the programs which best fit their needs. 

Participants indicated, however, that the IU had too many 
separate pieces and that certain sections needed simplification. 

The data from the Main Field Test were not as conclusive as 
the staff had hoped, however, and it was decided to retest the 
objectives at the same time the operational testing of the product 
was done. 

Main/Ope rational Form Development. Following the Main Field 
Test, the product was prepared for operational use. The revised 
form included: 



Preface : This included in one booklet: (a) an introductory 
section, introducing users to trends in secondary social studies 
and describing how to use the IU; (b) a screening aid, to help users 
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decide which programs in the IU to review in detail; (c) a preview 
chart, summarizing major features of all programs; (d) program 
previews, or summaries of each program; and (e) a glossary, defining 
the characteristics listed in the screening aid. 

Reports: Updated and slightly revised reports were prepared. 

The package was boxed for mailing; it included general directions 
for use on the side of the box. 

Combined Main/Operational Field Test (M/OFT). A combined 
Ma^n/Operational Field Test was conducted in the spring of 1970. 

Two groups were selected to review the IU under main or performance 
conditions, i.e., with laboratory supervision; and two groups 
reviewed the unit under operational conditions, i.e., on their own. 
Twenty-three secondary schools in Iowa were matched on the basis of 
number of decision-makers, formality of decision-making group, 
authority of decision-making group, and experience with the pro- 
grams described in the unit. Half of the schools were randomly 
selected as main or performance sites and half as operational 
sites. The other two groups were selected to balance the possible 
uniqueness of the all-Iowa schools. Sites were chosen to represent 
southern, eastern, and western states, and to be replications of 
the Iowa study. Assignment of these non-Iowa schools to the Main 
Field Test conditions was uncontrolled. These participants were 
given a choice of conditions. (This was necessitated by the added 
time and expense required for a Performance Field Test; the staff 
could not provide the necessary control at the widely separated 
non-Iowa sites. ) 



TABLE 2 

Number of Subjects in Main/Operational Field Test 





Operational 

Subjects/Sites 


Performance 

Subjects/Sites 


Iowa 


21/12 


22/11 


South 


2/1 


0/0 


East 


16/11 


0/0 


West 


9/3 


15/8 



During this combined Main (MFT) and Operational (OFT) Field 
Test, "main" subjects reviewed all parts of the IU and followed 
review instructions, including the requirements that they review 
all programs through the "summary" level and that they read at 
least three Reports. With the Operational Field Test sites, no 
controls were exerted over whether the schools used the Information 
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Units or not; nor were controls exerted over how they were used. 

The only conditions for operational use were the prior agreement 
to return the Information Unit by mid-June, to complete a questionnaire, 
and to permit a Laboratory staff person to conduct an interview 
with a representative of the group. No Laboratory representative 
was present at or participated in the use (or non-use) of the 
package. This absence of control was purposeful; the intent of the 
Operational Field Test is to determine if the product could "stand 
on its own" as a mailable, self-explanatory unit. 

A detailed report on the results of this combined Main/ 

Operational Field Test can be found later in this report. 

Final Product Revision After the Operational Test . The Preface 
was revised in line with suggestions from field test users; in 
particular, the screening aid was greatly simplified. In addition, 
the reports have been updated. Because of publishing constraints, 
the Information Unit was published as a single 8-1/2x11" bound volume. 

6.4 What evaluation has been carried out at the various stages of 
the project's development ? 

Three major field tests of the American Government Information 
Unit were conducted. The first, the preliminary test, a feasibility 
test, was largely a non-quantitati ve investigation relying heavily 
on staff judgment. The results, in effect, convinced the staff 
that there was a need for the product and that the general format 
proposed probably would fill the need. 

The Main and Operational Field Tests were empirical, quanti- 
tative studies conducted to determine whether the Information Unit 
could, in fact, meet the objectives set for it. These objectives 
were of three types: (1) cognitive objectives — knowledge, compre- 
hension, application, and evaluation of information; (2) enaoling 
objectives— intermediate objectives believed necessary for the unit 
to meet cognitive and affective objectives; and (3) affective 
objectives — value to and liking of the Information Unit by users. 

Each objective was defined in terms of several specific objectives 
that could be measured in terms of user performance (behavioral 
objectives) and expert judgment; conditions for testing were speci- 
fied and tentative standards set. (See Chapter 1.0 above for more 
detail.) 

Two significant questions that could have been asked but were 
not are these: First, do school people adopt programs as a result of 
using the Information Unit? No answer to this question has been 
provided because the purpose of the Information Unit was to enable 
people to consider these new programs rationally — with rejection 
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or adaptation being equally viable courses of action. Second, does 
the Information Unit enable school personnel to select programs 
(curricula) that will result in the most student learning? An 
answer to tl.is question was not sought because of two constraints: 
(1) at least two years would be required to study the effects on 
students of a single curriculum selection and (2) at least quasi- 
experimental conditions would be required to simultaneously study 
the effects of rejected alternatives in order to provide such a 
comparative test of the effect of not using the IU. 

Results of the evaluation of the combined Main/Operational 
Tests are summarized below. On the basis of these results, it was 
concluded that the objectives were successfully met. A detailed 
report of results can be found in the Appendix to this report. 
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*Not tested with MFT-Other and OFT-Other samples. 
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Summary of Results 
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Conclusion 

Objective Test Standard Result Pass Fail 
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6.5.1 Was there any revision made In the product from the time 

of Its final evaluation to Its ger Tal relea se? ~ 

The Preface and Reports were revised and updated, and the unit was 
packaged as a single bound volume. 

6.5.2 What effect did these revisions have? 

This has not yet been ascertained. Presumably, since much revision 
was based on earlier evaluation data, it will improve the effectiveness of 
the unit. 

6.6 Have any independent evaluations been made ? 

None have been reported. 

6.7 What evaluation is still being conducted or is planned for ? 

No specific additional evaluation is anticipated, but the 
developer plans to continue research and development in the area 
of curriculum information systems and analysis. 
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7.0 PROJECT HISTORY 



7.1.1 What is the name of the educational product ? 

The American Government Information Unit. 

7.1.2 By what other names or acronyms is (or has) the program 
been known? 

Originally* the unit was termed "Integrated Information Unit." 
It was often called the "IIU" for short. It was also called the 
Information Unit on Secondary Social Studies Programs. 

7.1.3 What organizations are responsible for the development 

o f the program ? ~ 

The Communication Program of the Far West Laboratory for 
Educational Research and Development. 

The address is: 1 Garden Circle, Hotel Claremont, Berkeley, 
California. 

7.1.4 Who Is in charge of the project ? 

Dr. C. L. Hutchins is in charge of the Information System 
Component responsible for the development of the Information Unit. 

Dr. John Hemphill is Director of the Far West Laboratory for 
Educational Research and Development. 

7.2.1 In what year was the project initiated ? 



1968. 



7.2.2 In what year will the project be concluded ? 

The major development of the American Government Information 
Unit will be concluded during 1970; a monitoring and updating 
activity will continue indefinitely. 

7.3.1 What is the principal source of funding ? 

Title IV of the , Elementary and Secondary Education Act; admini- 
stered by the Bureau of Research, the U.S. Office of Education. 
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7.3.2 What is the total investment that has been made in this 
project ? 

$185,000. 

7.4 Who is the commercial publisher ? 

Lockheed Educational System. 

7.5 Who are people who have had major roles in the project 
development? 

Credit must go to staff of the Information System Component of 
the Communication Program of the Far West Laboratory for Educational 
Research and Development. Those most closely associated with the 
development are: Nancy Adelson and Sandra Crosby, who comprised the 
project development team; Linda Sikorski, who directed evaluation; 
Diane Howland and Fannie Walton, who supervised field tests; Jean 
Sims, who supervised production; C. L. Hutchins, Project Director; 
and Paul D. Hood, Director of the Communication Program of the Far 
West Laboratory for Educational Research and Development. 
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APPENDIX 

MAIN/OPERATIONAL FIELD TEST RESULTS 



Results of the evaluation of the combined Main/Operational 
Tests are as follows: 

1. Cognitive Objectives: Does the IU provide knowledge, 
comprehension, application, and evaluation of information about 
American government programs? The field tests indicate that it 
does. The performances that were required during the Main Field 
Test (MFT) and Operational Field Test (OFT)® and the results of this 
testing are as follows: 

a. (Knowledge): After using the IU, subjects will, without 
guessing, correctly identify as true or false significantly more than 
50% of a set of statements describing major characteristics of 
programs. The statements were derived from lists which contained 
major points report writers were trying to get across about the 
programs. (MFT subjects were required to read at least three program 
reports; OFT subjects were not required to read any specific number 
of reports.) Subjects were asked to respond to items regarding 
the three programs with which they felt most familiar. The subjects 
were instructed not to guess at answers; rather they were to indicate 
"don't know" wherever they were not sure of an answer. To determine 
whether subjects scored significantly higher than 50% correct, the 
null hypothesis, u=50, was tested against the research hypothesis, 
p> 50. All sample means are significantly higher than 50% correct 
(MFT-Iowa, t-3.4, p<.005; OFT-Iowa, t=1.72, p<.05; MFT-Other, t=4.95, 
p<. 0005; OFT-Other, t=4.57, p<.0005. 

Thus, after using tie IU, Field Test sample means were higher 
than 50% correct. This is reflected in the confidence intervals 
we can compute for each sample. 7 They are as follows (based on 



®These include matched OFT and MFT samples in Iowa and MFT and OFT 
samples in western, eastern and southern states. See below for 
description of these samples. Sample sizes are: Iowa MFT, n=22, 
Iowa OFT, n=21 , non- Iowa MFT, n=15; non- Iowa OFT, n=27. Reports 
of results may occasionally be based on slightly smaller sample 
sizes since there were sometimes non-respondents for particular 
questions or measures, particularly at the operational sites. 

?The level of confidence is the probability that a sample interval 
will include the population mean. Thus, it is 90% "sure" or 
"confident" that the above four intervals do include the means of 
the corresponding populations. 
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90% level of confidence): 



MFT-Iowa 58.62 - 69.72 
OFT-Iowa 51.28 - 59.92 
MFT-Other 66.36 - 78.64 
OFT-Other 61.57 - 71.07 



It should be noted that the lower limit of each interval is 
above 50%. 

Results did not change when schools rather than individuals 
were used as the unit of analysis. 

Over 70% of subjects in each of the two MFT and two OFT samples 
achieved scores of 50% or higher. Mean scores are as follows (standard 
deviations are indicated in parentheses): MFT-Iowa, 64.17% ( S=1 7.7); 
OFT-Iowa, 55.60% (S=14.55); MFT-Other, 72.5% (S=17.02); OFT-Other, 
66.32% (S-15.56). 

MFT subjects completed "Before" and "After" knowledge tests. 

The knowledge increase among Iowa MFT subjects was significant 
( t= 1 2.52, p<.0005). 8 

b. (Comprehension): After using the IU, subjects will receive 
scores significantly higher than 50% correct when indicating agree- 
ment or disagreement with statements which represent implications 
based on facts about programs they reviewed. (MFT subjects were 
required to read at least three program reports; OFT subjects were 
not required to read a specific number of reports.) 

As discussed earlier, it was necessary to reject the null 
hypothesis, y=50, to determine if subjects performed significantly 
higher than standard. Again, they did. Results for MFT-Iowa are 
significant (t=3.6, p<.005); results for OFT-Iowa subjects are 
significant (t=1.69, p< . 1 0 ) ; and for MFT-Other (t=4.95, p< . 0005 ) ; and 
for OFT-Other (t=2.43, p< . 025 ) . 



Confidence intervals are as follows, using .90 as the level 
of confidence: 



MFT-Iowa 

OFT-Iowa 

.MFT-Other 

OFT-Other 



56.47 - 63.97 
51.28 - 60.82 
56.94 - 72.88 
54.58 - 65.64 



8 MFT-non-Iowa "Before" scores were too spotty for this analysis. 



70% or more of each sample received scores as high or higher 
than 50%. Mean scores were as follows (standard deviations are in 
parentheses): MFT-Iowa, 60.22 (S=ll .97 ) ; OFT-Iowa, 56.05 (S=16 .08) ; 
MFT~0ther , 64.91 (S=19.31); OFT-Other, 60.11 (S=18.'I5). 

MFT-Iowa subjects completed Before and After measures of com- 
prehension. The comprehension increase among MFT-Iowa subjects was 
highly significant (t=14.78, p<.0005); among non-Iowa MFT subjects, 
it was also significant (t=5.62, p<.0005). When scores are compared 
for the matched Iowa MFT and OFT schools (MFT-Before vs. OFT-After), 
there is an increase from MFT-Before to OFT-After which is significant 
(t=7.17, p<. 0005) . 

c. (Application): After using the III, subjects will indicate 
how well each III program meets their five most important needs and/or 
constraints. They will also indicate a decision (adopt, pilot test, 
reject, or send for sample materials) for each program. Subjects' 
decisions about each program will be consistent with their estimates 
of how well each program meets their needs. That is, if a subject 
reports that a program meets his needs, he will decide to investigate 
the possibility of adopting it; otherwise, he will reject it. 

Field test subjects were asked two questions. First, they were 
asked to indicate for each program they reviewed whether it met none, 
some, or all of their five most important needs. Second, they were 
asked to indicate a decision for each program they reviewed, i.e. 
whether they would adopt or pilot test all or part of it, reject it, 
or send for sample materials regarding it. Subjects' responses to 
the two questions were considered together such that each subject 
indicated one of the following response combinations for each program 
he reviewed: 

(A) "The program meets none or less than half of my needs and 
I would reject it." 

(B) "The program meets none or less than half of my needs and 
I would send for sample materials." 

(C) "The program meets none or less than half of my needs and 
I would pilot test or adopt all or part of it." 

(D) "The program meets all or more than half of my needs and 
I woul d reject it." 

(E) "The program meets all or more than half of my needs and 
I would send for sample materials." 

(F) "The program meets all or more than half of my needs and 
I would pilot test or adopt all or part of it." 

These six response combinations can be represented in Table 3. 



TABU. 3 

I would I would send I would pilot test 

reject for sample or adopt all cr 

program materials part of the program 



Program meets 
none or less than 
half of my needs 

Program meets 
all or more than 
half of rr\y needs 

To assess whether subjects 1 decisions were consistent with 
their needs, certain predictions were made regarding the relative 
number of occurrences of the six possible response combinations. If 
the letters in the table are replaced by the number of occurrences 
of the corresponding response combinations, the following should be 
true: 



A 


B 


c 


1 

1 ° 


LU 


F 



The 


number 


in 


cell 


F 


wi 1 1 


be 


greater than 


the 


number 


in 


D. 


The 


number 


in 


cell 


A wi 1 1 


be 


greater than 


the 


number 


i n 


C. 


The 


number 


i n 


cell 


F 


wi 1 1 


be 


greater than 


the 


number 


in 


C. 


The 


number 


i n 


cell 


A will 


be 


greater than 


the 


number 


in 


D. 


The 


number 


in 


cell 


E 


wi 1 1 


be 


greater than 


the 


number 


in 


B. 



To elaborate, where subjects indicate that a program meets all 
or most of their needs, they should be more likely to adopt or pilot 
test it than to reject it; where subjects indicate it meets few or 
no needs, they should be more likely to reject it. Similarly, subjects 
who feel programs meet their needs should be more likely to send for 
materials or adopt or pilot test than subjects who feel programs 
don't meet their needs; however, they should be less likely to reject. 

Results were tallied for Iowa subjects. Each respondent made 
one of the six responses for each program he reviewed. Thus, the 
number of responses for any one subject is equal to the number of 
programs he reviewed. For 21 MFT respondents, there were 64 respon- 
ses altogether; for 16 OFT respondents, there were 59 responses." 

Tables 4A and 4B show the numbers of occurrence of each of the six 
response combinations over the two Iowa samples. 



^Non-respondents were subjects who responded only to the decision 
question and did not respond to the needs question. 




41 

35 



TABLE 4A 



MFT- Iowa 





I would 


I would send 


I would pi lot test 




reject 


for sample 


or adopt all or 




program 


materials 


part of the program 


Program meets 


~~ A r 


B 


C 


none or less than 
half of my needs 


6 


0 


6 


Program meets 


D 


E 


F 


al 1 or more than 
half of my needs 


1 


24 


27 


- ----- 








TABLE 4B 






OFT 


-Iowa 






I would 


I would send 


I would pi lot test 




reject 


for sample 


or adopt all or 




program 


materials 


part of the program 


Program meets 


A 


B 


C 


none or less than 
half of my needs 


5 


4 


2 


Program meets 


D 


E 


F 


al 1 or more than 


2 


37 


_ . 10 .. ! 


half of my needs 







Thus, over the OFT sample, all the predictions were borne out; 
over the MFT sample, all but one of the predictions were borne out. 
Among MFT subjects, the number of A occurrences was equal to rather 
than greater than the number of C occurrences. That is, subjects 
who felt programs met less than half of their needs were as likely to 
adopt or pilot test them as to reject them. However, the prediction 
that they would more often reject them was somewhat weaker than the 
other predictions made, since it presumed that a program meeting only 
one or two needs, even if they were important ones, would not be worth 
pilot testing all or part; obviously, this would not always be the case. 

Since the above tables are biased in favor of subjects who re- 
viewed and responded for more programs, a method was devised to choose 
just one program per subject. 

Each subject responded for at least one program. If he responded 
for only one, that response was tallied. If he responded for more 
than one, the response for the second, or third, or fourth program he 



reviewed was tallied in accordance with the plan: choose a program 
which is different from that chosen for the preceding subject and 
the following subject. Over the 21 MFT respondents and the 16 OFT 
respondents, the responses shown in Tables 4C and 4D occurred. 

TABLE 4C 

MFT-Iowa 

I would I would send I would pilot test 

reject for sample or adopt all or 

program materials part of the program 



Program meets 
none or less than 
half of my needs 



Program meets 
al 1 or more than 
half of my needs 



TABLE 40 
OFT-Iowa 

I would I would send I would pilot test 

reject for sample or adopt all or 

program materials part of the program 



Program meets 
none or less than 
half of my needs 



Program meets 
all or more than 
half of rny needs 



Results of the analysis depicted in Tables 4A and 4B were repli- 
cated. It was concluded that subjects' decisions were consistent with 
their needs. 

. d. (Evaluation): Having used the III, 50% or more subjects 
will decide on a course of actionlO regarding at least six of the 
nine programs. Here a puzzling result was noted. On the one hand, 
MFT-Iowa subjects, MFT-Other subjects, and OFT-Other subjects 



) 



A 

3 


B 

0 i 


1 C 

0 


D 


E 


F 


o 


9 


4 

■ 
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B 


C 


3 


0 


3 


I D 


E 


F 


L_ 


8 


6 



^Reject, send for materials, pilot test, or adopt. 




43 

37 



passed easily: over 60% made decisions for at least six of the nine 
programs. However, only 38% of OFT-Iowa subjects made as many as 
six decisions. Table 5 shows these results. 

TABLE 5 

Percent of Subjects Making Decisions 
forO, 1, 2, 3, 4, 5, 6, 7, 8, or 9 programs 



No. of MFT-Iowa N=21 OFT-Iowa N=21 OFT-Other N=27 MFT-Other N=14 



programs 


% Ss. 


Cum. % 


% Ss. 


Cum. % 


% Ss. 


Cur,. % 


% Ss . 


Cum. 


9 


62 


62 


3." 


33 


47 


47 


<10 


40 


8 


9 


71 


0 


33 


10 


57 


13 


53 


7 


0 


71 


5 


38 


3 


60 


7 


60 


6 


0 


71 


0 


38 


0 


60 


0 


60 


5 


0 


71 


0 


38 


7 


67 


0 


60 


4 


0 


71 


5 


43 


3 


70 


0 


60 


3 


24 


95 


33 


76 


13 


83 


20 


80 


2 


0 


95 


14 


90 


3 


86 


13 


93 


1 


5 


100 


5 


95 


3 


89 


0 


93 


0 


0 


100 


5 


100 


10 


99 


7 


100 



If the percent making a decision for each of the nine programs 
is considered, it is found that a majority of all samples--even OFT- 
Iowa--make decisions for at least sever programs. Table 6 shows 
these results. 
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TABLE 6 



Percent of Subjects making decisions for each program 





MFT- 


Iowa N=21 


OFT- 


Iowa N=21 


OFT- 


Other N=27 


MFT- 


Other N=14 


Program 


Dec. 


No Dec. 


Dec. 


No Dec. 


Dec. 


No Dec. 


Dec. 


No Dec. 


SRSS 


85% 


1 5%* 


57% 


43% 


77% 


23% 


73% 


27% 


Amherst 


76 


24 


52 


48 


70 


30 


73 


27 


Law 


81 


19 


57 


43 


67 


33 


53 


47 


HSC 


71 


29 


52 


48 


70 


30 


73 


27 


ERC 


76 


24 


48 


52 


63 


37 


53 


47 


HSSC 


71 


29 


67 


33 


67 


33 


60 


40 


LFC 


81 


19 


52 


48 


70 


30 


73 


27 


Harvard 


90 


10 


57 


43 


77 


23 


73 


27 


USU 


90 


10 


43 


57 


60 


40 


67 


33 



*The percentages for a given set of subjects add to 100% because 
subjects were asked to make "decisions" about all of the programs in 
the information unit— regardless of the number of programs they studied 
in "detail" by reading the Reports. 



At first glance this seems paradoxical; however, such a result occurs 
because OFT-Iowa responses distribute themselves fairly evenly over 
all programs. That is, subjects making only two or three decisions 
tended to make them for different programs, such that for seven of 
the programs, about half of the sample made decisions. 

The evidence supports the IU as a decision-making tool; in 3 of 
4 samples, users could make at least 6 decisions, and in all 4 
samples, a majority made decisions about most or all programs. 

In asking why OFT-Iowa subjects performed so poorly relative 
to the other groups, the patterns of use recorded by subjects was 
analyzed. It was found that while all MFT subjects used the 
"Screening Aid" (they were required to) and while all but one of 
the OFT-Other sample used it, only half (11 of 21) of the OFT-Iowa 
sample used it. Table 7 was derived from the data on 79 subjects 
who reported their patterns of use. 





TABLE 7 



Use of Screening Aid as Related to Number of Decisions Made 



Used 

Screening Aid 

Did not use 
Screening Aid 



Made fewer than Made 6 or more 



6 decisions decisions 



1 


44 


8 


3 



Since all but one of the non-users of the Screening Aid were OFT- 
Iowa subjects and since the distribution of these non-users between 
those making more than six decisions and those making less than 
six was disproportional to the trend at large, we are free to 
conclude that the use of the Screening Aid was regularly accompanied 
by an increase in the number of decisions made. Whether the use 
of the Screening Aid accused, the increased decision making cannot 
be determined. However, it seems warranted to recommend the use of 
the Screening Aid to all users. 

Why is the Screening Aid useful --but not used? By design, the 
Screening Aid is meant to enable users to reject programs from con- 
sideration without having to review them, and there were data from 
MFT-Iowa subjects that they could, in fact, reject programs after 
using the Screening Aid. Further evidence that the Screening Aid 
fulfilled its function was also reflected in the final decisions 
made by subjects. Those who used the Screening Aid made propor- 
tionately more "reject" decisions and fewer of the indefinite "write 
for sample materials" decisions. Thus, it is possible that the 
Screening Aid (or some similar device) is necessary to help subjects 
make decisions. If they have to depend on the Reports alone, they 
probably are faced with too much information to sift through it 
efficiently. The main strength of the Screening Aid seems to be 
that it helps subjects to make a few definite decisions and to 
decide which of the nine reports to read. 

Unfortunately, the utility of the Screening Aid is not obvious. 
This conclusion is supported by results from MFT-Iowa subjects 
when they were asked to indicate agreement with the following 
statement: "I wouldn't use the Screening Ai'd unless directed to 

do so." Forty- two percent indicated that they agreed with the 
statement and 16% indicated they neither agreed nor disagreed. 

Thus, if this sample had not been required to use the Screening 
Aid, results might have been close to those obtained for the OFT- 
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Iowa sample. About 50% of the MFT-Iowa sample also agreed with the 
following: "The directions for the Screening Aid were too compli- 

cated to follow," or "The Screening Aid was awkward to use." Thus 
the utility of the Aid must be made clearer, and its ease in use 
improved. 



2. Affective Objectives: Is the IU attended to, responded 
to, and valued? The field tests indicate that it is. Required 
performances and results are as follows: 

a. Given the opportunity and a reason to use the IU, subjects 

will attend to it, and will become involved with it. This will 
become apparent via a number of unobtrusive measures: Attrition : 

There was less than 10% attrition of the Iowa sample from beginning 
to end of the field test; Co operation : Over 70% of the questionnaire 
items were completed by 80l~ or more of the Iowa subjects and the 
non-Iowa subjects; Recognition of parts : Over 80% of the Iowa sam- 
ples were able to indicate some extent of agreement (as opposed to 
"don't know" or "neither agree or disagree" responses) with statements 
made about the various pieces (Screening Aid, Glossary, Reports, 
etc.) of the IU; Recommendation : When asked if they would recommend 
that their school use an expanded version of the IU, over 90% of 

all four samples said "Yes." When asked if they would recommend 
IU's in other secondary subject areas, over 90% of three of the 
four samples said "Yes," and 85% of the MFT-Iowa sample said "Yes;" 
Time Spent : MFT subjects had to spend a predetermined amount of 
time with the Preface; OFT subjects could use it as they pleased. 
Among 34 subjects in the two OFT samples for whom we had relevant 
data, 29 spent over 45 minutes with the Preface; 13 of these spent 
over an hour. Of the 34, the majority (31) spent over an hour with 
the Reports; the majority of these (18) spent over two hours with 
them, and seven subjects spent over four hours. 

It was felt that involvement with and attention to che IU were 
indicated by the above five observations. 

b. Given a list of possible sources of information for getting 
adequate information about new programs, subjects will indicate that 
the IU is among the most useful and easy to use and is more useful 
and easy to use than all "secondary" sources, i.e., sources not 
providing direct contact with programs or colleagues. MFT and OFT 
subjects rated a number of sources, including the IU, on seven-point 
semantic differential scales of "useful-useless" and "easy to use- 
difficult to use." Tables 8 and 9 show results of these ratings 
over the four subject groups. 
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TABLE 8 





Mean Ratings by Subjects 


on "Useful" 






MFT-Iowa 


OFT- Iowa 


MFT-Other 


OFT-Other 


Journals* 


5.06 


5.43 


5.57 


5.48 


Conversations 


5.95 


6.05 


5.64 


6.22 


Conventi ons* 


4.48 


4.25 


4.64 


5.58 


Consul tants* 


4.86 


4.95 


4.64 


5.56 


Publishers* 


4.62 


4.65 


4.14 


5.26 


Site visits 


5.95 


5.57 


6.43 


6.50 


Workshops 


5 .62 


5.81 


5.93 


6.30 


Pilot projects 


5.71 


5.45 


6.14 


6.31 


IU 


5,33 


5.67 


5.43 


6.26 


TABLE 9 




Mean Ratings 


by Subjects on 


"Easy to Use" 






MFT-Iowa 


OFT- Iowa 


MFT-Other 


OFT-Other 


Journals* 


4.19 


4.71 


4.93 


5.56 


Conversations 


5.67 


5.05 


5.86 


5.58 


Conventions* 


3.95 


3.05 


3.79 


4.65 


Consul tants* 


3.38 


4.20 


3.79 


4.24 


Publishers* 


4.10 


4.05 


3.43 


5.15 


Site visits 


3.48 


4.29 


3.71 


4.42 


Workshops 


3.71 


4.95 


4.43 


5.15 


Pilot projects 


3.86 


3.90 


4.57 


5.27 


IU 


5.05 


5.33 


4.29 


5.89 



♦"Secondary" source: Indirect contact with the project. (Primary 
sources include interpersonal and direct contact with the project.) 



TABLE 10 



Mean Scale Values 


for Sources, Useful 


x Easy to Use 




• 


MFT-Iowa 


OFT-Iowa 


MFT-Other 


OFT-Other 


Journals 


21 .43 


26.52 


28.50 


31 .00 


Conversati ons 


34.67 


30.38 


33.79 


34.67 


Conventions 


18.33 


14.25 


18.50 


27.88 


Consultants 


16.76 


21 .90 


19.36 


25.40 


Publishers 


19.29 


20.45 


15.93 


28.41 


Site visits 


20.86 


24.95 


24.50 


29.08 


Workshops 


21 .86 


29.76 


28.64 


31 .81 


Pilot Projects 


22.95 


21 .80 


28.50 


32.85 


IU 


28.48 


31 .05 


26.29 


37.37 



3. Enabling Objectives: Is the IU a feasible means for 
reaching stated terminal objectives? 

Many of the following criteria are judged "successful" by 
subjective and admittedly biased judges. With additional time and 
money all could have been submitted to empirical, quasi-experimental 
confirmation. Because they were defined as enabling or subordinate 
objectives, however, it was thought that the detailed field testing 
of these objectives would be appropriate only if the terminal 
(cognitive or affective) objectives were not met. 

a. Parallel treatment: Programs will be treated fairly and 
in a parallel manner. The perception of developers will be most 
important in judging successful achievement. 

Presence of parallel treatment was confirmed by asking the 
developers of the nine programs whether the use of a standard format 
for the Reports caused the omission of any important information 
about their programs. None of the nine felt that any important 
material had been omitted. 

b. Accuracy: The information provided to the user must be 
accurate. Measurement can be provided by seeking the judgment of 
the developers. 
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Following the OFT, each developer was asked whether the report 
on his project was accurate. If not, he was to list any inaccuracies. 
Developers found the report accurate; noted minor corrections to be 
made; noted major corrections to be made. For the most part, these 
corrections -consisted of updatings or the addition of information 
which had not been available when the OFT version of the reports 
was written. The corrections were incorporated into th<- final 
version of the IU. 

c. Consistency: Consistency is closely related to accuracy; 
more specifically, it concerns the need t"> say the same things in 
the same way at the several levels of inf. "(nation. 

Consistency was tested by asking the developers to indicate 
errors in all of the levels of information. Only one of them 
mentioned a discrepancy (two different warehouses of the publisher 
were given as the correct address). An appropriate correction was 
made. 



d. Objective treatment: All statements of fact must be 
supported by the ability of a number of judges to look at the same 
information and draw the same conclusions about its nature. The 
developers should not perceive any bias in the IU. 

This objective was estimated by asking the developers whether 
they had noted ar\y bias in the report on their program or in the 
reports on the other eight programs. No one reported any bias in 
any of the reports. Also, all Iowa subjects who reviewed programs 
they were already familiar with indicated that the IU had given a 
fair treatment of those programs. 

e. Timeliness of information : The information in the unit 
must be up-to-date. The time span between collection of information 
and its use must not sufficiently diminish the value of the infor- 
mation. This will be measured by the judgment of staff. 

The developers were asked to send all updating information 
about their programs by August 15, 1970. This information was 
added to the program reports from then until October 30. In addi- 
tion, the developers were called in the latter part of September 
in order to ascertain whether there ere any further developments 
which should be reported on. As the unit is expected to be avail- 
able to teachers by January 1, 1971, it was agreed that this objec- 
tive had been met. 

f. Flexibility: Because the IU will have multiple users and 
these users will have different patterns for reviewing the IU, it 
is important that the IU format be flexible. 
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Users noted throughout the tests that they liked the flexible 
format of the field test versions of the IU (background information, 
instructions, and summaries of the programs were printed in one 
booklet; the nine reports comprised separate booklets). Cross- 
tabulations -of background and demographic variables with criterion 
variables indicated no significant differences across different 
user groups and situations. 

g. Relevance: Those areas of highest priority or relevance 
should be treated first. Measurement should be achieved by the 
judgment of consultants, users, and staff who compare the IU's 
priorities against state and local priorities. 

All groups agreed that the focus of this IU on well-developed 
American government programs was justified. American government 
is the one social studies subject which must be taught in every 
state, usually by law. This objective, therefore, was achieved. 

h. Uniqueness: The IU must meet its objectives as well as 
or better than any similar secondary sources of information about 
American government programs. Measurement of this objective must 
be largely a matter of expert judgment and user opinion. 

There is no comparable secondary source devoted to description 
of programs suitable for American government courses. The sources 
of general information about social studies programs, which could 
be considered comparable, are: (a) publishers' brochures and pam- 
phlets describing student and teacher materials; (b) the analysis 
of programs done by the Social Science Education Consortium in 
Boulder; (c) the April, 1970, issue of Social Education , which 
contained brief analyses of new social studies projects ;12 (d) the 
Marin Social Studies Project; (e) workshops and conferences about 
the new programs; (f) presentations by publishers' representatives; 
(g) publications of the various social studies councils; (h) the 
ERIC Clearinghouse of Social Science Education; and (i) the EPDA 
Civics Dissemination Institutes conducted by Indiana University. 

All of these sources were judged to be somewhat less effective 
than the IU for one or more of the following reasons: 

1. Information overload (e, h) 

2. Inaccessability to users (d, e, i, possibly h) 

3. Incomplete information on the programs suitable for 
American government classes (a, b, c, d, e, f, h, i) 




^The authors' 
IU Reports. 



evaluative comments are being incorporated into the 
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/ i. Incomparable treatment of all programs (a, e, f, i) 

5. Not teacher oriented (h) 

6. Difficult to use, e.g., restricted access, not compact 
(d, e, 'r, i ) 

7. Costliness (e) 



i. Cost-effectiveness: After using the IU, subjects will 

respond to cost and reward scales; the average values of reward and 
cost will be such that reward/cost is greater than one. 

Seven-point scales were used for subjects to estimate whether 
the effort, time, and expense of using the IU were acceptable or 
unreasonable, and whether the help, personal benefit, and enjoyment 
from using the Unit were superlative or minimal. Table 11 illustrates 
average scores on each scale (see next page). The average of the last 
three scores was divided by the average of the three cost scores to 
compute reward/cost. Tested with OFT-Iowa subjects, the average reward 
value was 5.61 and the average cost value was 3.62; the obtained quotient 
was 1.55. The IU was rated highest on the reward factor of "helpful- 
ness," and lowest on "personally beneficial;" the most extreme cost 
factor was financial cost and the least, for time cost. 

Other Relevant Evaluation Information: In addition to 
the other evidence presented, the following pieces of information 
seem relevant to the evaluation of the American Government IU: 

a. MFT and OFT results were compared directly to see if 
different results were produced by the two types of testing conditions. 

On two measures, knowledge and evaluation, MFT-Iowa schools pf - 
formed above the matched OFT-Iowa schools. The factor which may 
have made the difference on the evaluation objective has already 
been discussed, i.e. that OFT subjects did not use the Screening 
Aid. For knowledge scores, there are two plausible explanations. 

First, MFT subjects had both Before and After tests and may have 
been demonstrating, in part, a testing effect. Second, MFT subjects 
completed After tests immediately after using the materials while 
for OFT subjects there was a delay usually of several days or weeks 
before follow-up testing could occur. Scores for OFT subjects may 
be considered to represent longer-term recall of information about 
programs, and these scores are predictably Tower. Thus, MFT con- 
ditions favor higher knowledge scores and evaluation scores; however, 
the knowledge difference is very likely an artifact of testing 
while the evaluation difference represents a very real necessity 
for improving the attractiveness of what appears an important com- 
ponent, the Screening Aid. 
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Average Ratings on Six Cost and Reward Items (OFT-Iowa) 



The IU was not 
helpful (for my 
purpose). I’d 
have been better 
off if I hadn't 
used it at all. 



Reasonable amount 
of effort. 



I would rather not 
have gone through 
the experience of 
using the IU. 

The IU is a time- 
saver; ordinarily, 
I would have to 
spend a great deal 
more time to get 
the same results 
that I got using 
the IU. 



The IU afforded 
me no personal 
benefit. 




The IU was very 
helpful (for my 
purpose). I'm 
much better off 
(in regard to my 
purpose) than I 
was before. 



An unreasonable 
amount of effort 
is requi red to 
use the IU. 

I very much enjoyed 
the experience of 
using the IU. 



Using the IU 
requires more time 
than it is worth; 

I would be better 
off spending the 
same amount of time 
reviewing programs 
without the aid of 
the IU. 

As a result of 
using the IU, I 
have benefited a 
great deal . 
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The IU is well 
worth this price. 



Too expensive. 



b. The data on "source preference" can be analyzed to compute 
a "consensus" score reflecting the degree of agreement among respon- 
dents regarding sources they would prefer to the IU. On the basis 
of the number of respondents who regard a source as either better 
than or not as good as the IU in regard to its usefulness and its 
"ease of use" for curriculum decision-making, we can compute an 
algebraic score for a source which more precisely locates it in 
relation to the IU. From the individual ratings for a subject, we 
can tell which of the sources was more valuable to him than the IU. 
The "score" for a source is computed by assigning a "-1" to a source 
if it is rated lower than the IU, a "+1" to a source if it is rated 
higher than the IU, and a "0" to a source if it is rated the same 
as the IU. The sum of scores for each source (over the total respon- 
dents) is a positive or negative value, depending on whether more 
respondents like it better or not as well as the IU. Figure 1 
presents the results of such analysis on the field test data. As 
can be seen, over three of the four samples, the IU is more valued 
than all secondary and most primary sources. 
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FIGURE 1 
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c. When asked to indicate their familiarity with all IU programs 
after using the Unit, subjects Indicated slightly increased famili- 
arity as compared to scores indicated before using the Unit. There 
were Before scores for MFT subjects and After scores for all subjects. 
Average "familiarity scores over each Iowa sample are graphed in 
Figure 2. 



FIGURE 2 




As can be seen, while the After scores are higher, they are still 
very low. This was expected since all subjects were included in 
the familiarity scores for all programs, even though many had not 
read the relevant program Reports. The extent to which non -readers 
depressed the scores became evident when further analysis revealed 



that where subjects read all or part of the Report for a program per- 
ceived familiarity was generally high. Figure 3 is a graph depicting re- 
sults where subjects had read or had not read program Reports. 

FIGURE 3 
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